Papers with reward-model-based RLHF
Filtered Direct Preference Optimization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the impact of RLHF on text quality have focused on reward-model-free RL. |
| Approach: | They propose an extension of direct preference optimization to improve model performance by analyzing the quality of the preference dataset. |
| Outcome: | The proposed method improves the performance of models optimized with DPO over those optimized with reward-model-based RLHF. |